DOCUHEHT RESOME 



ED 084 268 



TH 003 276 



AUTHOR 
TITLE 

INSTITUTION 
PUB DATE 
NOTE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Christian^ Floyd T, 

Preliminary Report: Florida's Plan for Educational 
Assessment. 

Florida State Dept. of Education, Tallahassee, 

30 Sep 70 

6p. 

MF-$0,65 HC-$3,29 

Cost Effectiveness; Decision Making j *Educa.tional 
Accountability; ^Educational Assessment ; *Educa tional 
Improvement; *E valuation Methods; Formative 
Evaluation ; *Management Systems ; Models ; State 
Standards; Summat ive Evaluation ; Technical Reports 
Florida 



ABSTRACT 

A plan fo 
focuses upon three interd 
system: input, process, a 
allocation, and assignmen 
process concerns the appl 
in ways that will produce 
the behavior resulting fr 
presented that is to exte 
years. During Phase One — 
educational programs will 
monitoring the discrepanc 
In Phase Two-Cost Analysi 
developing cost data for 
specified. Phase Three — P 
determining ways of modif 
input (resources) and out 
discrepancies identified, 
the establishment of acco 



r educational as 
ependent aspects 
nd output. Input 
t of resources, 
ication and util 

changes in stud 
om the process* 
nd over a period 
Product Assess me 

be initiated an 
ies between "wha 
s (input assessm 
the state educat 
rocess Assessmen 
ying the functio 
put (student per 

The proposed pi 
untability for e 



sessment i 
of a maua 
encompass 
both human 
ization of 
ents or st 
A thre.e-ph 
of from f 
nt — a stud 
d instrume 
t is" and 
ent) , a me 
ion system 
t — will be 
nal relati 
f ormance) 
an is expe 
duca tional 



s presented which 
gement control 
es tjte provision, 

and material; 

these resources 
aff; and output is 
ase model is 
ive to seven 
y of output from 
nts developed for 
"what should b^." 
thod for 

will be 

concern^ed with 
onship between 
to reduce 
cted to lead to 

outcomei?. (DB) 



ERLC 



U S OEF'ARTMENT OF HEALTH. 
EDUCATION AMELFAHE 
NATIONAHNSTITUTEOr 
EDUCATION 

THIS DOCUMENT "^^^ PRELIMINARY RKPORTj 

?Mf-M?:Jjo^.^'io^^^^^ FLORIDA'S PLAN FOR EDUCATIONAL ASSESSMENT 

M INC. 11 MOINTSO* Vl!:>^ OH OI»l N I ONS 
SI A If D DO N01 NCCt:SSA«ILY Mflf^Ml; 
Sf Nl 0' -lOAL NAtlONAL iNiTITUlC Ot 
GDUCATIQN MOSmON OH POUCY 



CIO 



ERIC 



Introduction 



It has been adequately stated in the recent past Chat evaluation systems 
^ exist for the purpose of developing information to be used by management in 

the decision-making process. The plan for educational assessment presented 
in this preliminary report proposes to serve this purpose through establishing 
_^ three distinct but interdependent aspects of a management control system. If 

an assessment program is to provide information which leads management to 
CD make positive change, it must focus upon these aspects: input, process, and 

VjuI output ♦ 

Input encompasses the provision, allocation and assignment of resources, 
both human and material. Process concerns the application and utilization 
of these resources in ways which will produce changes in students (or staff). 
Output is the behavior resulting from the process (behavior is used here in 
a general sense to include any observable student attributes which are of 
concern to the schools) . 

Educational management is equally concerned with assessment of each of 
these aspects of the system. Input has traditionally received greatest 
emphasis and attention at the state level. Assuring that funds are appro- 

* priately expended, that instructional personnel are properly trained and 

assigned, and that facilities are adequate to their purposes has been, and 

- properly should be, a concern of state level management. 

Outputs have received less emphasis at the state level, but it should 
^'-^ -^v^ not be thought that output measures have not been gathered. Statewide testing 



programs are output measures. Tests and other instruments administered at 
district and school levels are output measures. So too are dropout rates, 
employment rates, and other measures of societal expectation with which the 
schools are concerned. 



Process has received the least attention at the state level. It is true 
that some revisions of process have received state attention, e.g. differentiated 
staffing, individualized instruction, and year-round school. However, these 
process modifications may be of little import when implemented without con- 
sidering desired or desirable changes in student behavior. Modifications 
must be evaluated in terms of their effects upon inputs and outputs. 

Related Activities 

The plan presented in the following pages builds upon several ongoing 
activities of the Department of Education. Further, the assessment program 
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can receive direct supporu from these activities. 

Developmenu" of product objectives has been a major part of state 
accreditation procedures for some time. The Research and Development 
program is providing assistance in development of cost analysis techniques 
and in the development of criterion-referenced achievement measures. The 
Joint Federal/State Task Force (iielmont Group) has made great strides in 
consolidating Federal reporting requirements; The Needs Assessment Project 
has compiled a great deal of data relating to the current status of education 
in Florida, and has available the analyticaJ. tools required in a comprehensive 
assessment program* Funds which may be used within the Department for staff 
development in planning and evaluation are available through Title IV of the 
Elementary and Secondary Education Act. These and other activities represent 
a preliminary planning, research, evaluation and development effort already 
under way in the Department, and will make a substantial contribution to the 
assessment program established under Chapter 70-399, Laws of Florida. 

The plan assumes the involvement of many agencies. Various units of 
the Division of Vocational Education and the Division of Elementary and 
Secondary Education, including the Florida Public Scltool Council, will pro- 
vide valuable input to the program. School district personnel will be 
invited to participate in formulation and validation of educational ob-- 
jectives, as will lay citizens, school board members, professional education 
organizations, private industry, university personnel, and other departments 
of state government. 



The Assessment Plan 

The plan proposed below is based on the premises that (a) the Department 
of Education is committed to a policy of positive change in the stat- system 
of education and (b) that this commitment extends to a redefinition of the 
state's role in the maintenance and control of the state system of education. 

The model presented is viewed as a three-phase model extending over a 
period of from five to seven years. It is felt that seven years is not an 
unreasonable time requirement for installation of a tested comprehensive 
assessment procedure* One of the features of the model is its flexibility 
and amenability to modification as circumstances change. Phase one, product 
assessment, will begin in 1971-1972; Phase two, cost analysis will also begin 
in 1971-1972. Phase three, process assessment, will necessarily begin at a 
later stage—probably 1972-1973. 

The logic for this sequencing is simple. If there is indeed dissatis- 
faction with system outputs, the extent of the dissatisfaction must be specified 
before corrective action can' be taker. Further, the costs currently allowed 
to the elements of the system must be known before money resources can be 
re-allocated. This reallocation also is dependent upon where the greatest 
dissatisfaction with output exists. 
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Phase One'— 'Product Assessment 

During this phase, a study of output from educational programs will be 

initiated, and instruments will be developed to monitor continously the 

discrepancies between "what is'* and "what .should be". Several steps should 
be started at once: 

1, The State Board of Education and other educational policy-makers 
will state and adopt goals for education. These goals will necessarily be 
broad, but must encompass the areas deemed by the policy-makers to be the 
concern of the state education system. It is to be emphasized that the goals 
are broad, relatively timeless, and perhaps visionary. They represent areas 
in which the policy-makers wish the school system to function; and to that 
extent they delimit the roles which the education system may play. In an y 
management-control-system it is essential that top management specify the 
scope of operation»or else control is impossible . 

2v Selected staff members of the Department of Education, from the 
Division of Elementary and Secondary Education and the Division of Voca- 
tional, Technical and Adult Education, will be assigned by their respective 
Division Directors to gather and consolidate measurable educational objectives 
from sources within and without the state. These objectives will specify 
desired outcomes for students and/or staff, the achievement of which can be 
determined. They will specify who is to change, the extent of change sought, 
and the conditions under which the change will be sought. 

During 1971-1972, the Division of Elementary and Secondary Education 
will focus upon general educational outcomes desired for children up to 
about the sixth grade level (approximately 5-12 years of age), while the 
Division of Vocational, Technical and Adult Education will focus upon voca- 
tional objectives for the elementary and junior high school and on at least 
three (3) specific occupational program areas at the high school level. 

3. Lay/professional boards will be empaneled to determine the relevancy 
of the collected objectives to state goals. Objectives determined to be 

not relevant to these goals will be modified or discarded. It should be 
stressed that curriculum per se is not an issue in this determination; 
the objectives will be curriculum-independent. (The importance of specifying 
relevant desired outcomes overshadows established curriculum organization.) 

4. Contracts will be let with competent professionals to develop and 
field-test items which measure cne achievement of the desired outcomes 
determined in the preceding step. 

5. Instruments constructed from the items developed in Step 4 will be 
administered to representative samples of students over the state. Since 
the list, of objectives is expected to be quite lengthy, item-sampling 
techniques will be applied. 
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6, Data obtained from the procedures of Step 5 will be uacd to assess 
status — the extent o£ performance deficiencies can be directly specified. 
These data will be of little use for specific recommendations on how process 
(curriculums , prof;ramn; etc.) can be modified. It can reveal which curric- 
ulum areas are in need of revision or modification, however. Further, these 
data can be integrated witli data currently available from the Department's 
Needs Assessment project to determine modifications in relation to the 
■educational environment (context) of a given district. 

Implicit in the forep,oing is the principle that each of the objectives 
will be logically derivable from the goala established in Step 1. That is, 
if an objective is irrelevant to an established goal, it is outside of the 
education system* s area of concern. 

Phase Two — Cost Analysis (Input assessment) 

During the cost analysis phase, a method for developing cost data tor 
the state education system will be specified. The following steps are seen 
to constitute this phase: 

1. Ascertain what cost breakdowns are desired, e.g., school costs, 
school costs by broad curriculum area, district cost by objective, or some 
combination of these or other breakdowns - 

2. Determine means for collecting these cost data, specifically con- 
sidering the *'changeover" period (if there is one) during which the old cost 
data reporting system will still be used. Steps 1 and 2 should use all 
available resources, including one or more task forces with legislative and 
professional representation. Compatibi? ity with Federal reporting requirements 
will be emphasized. 

3. Appropriate cost data collection instruments will be developed and 
designed to meet the requirements established in Steps 1 and 2. Installa-- 
tion of these instruments must be accompanied by extensive staff development 
activities for both district and department fiscal personnel. 

4. Uses to which cost data will be put should be clearly specified if 
collection of the data is to be justified- The relationship of these data 
(which represent input assessment) with product assessment and process 
assessment must be stressed. In order to derive measures of efficiency, or 
of cost/benefit, crosswalks between these three assessment phases must be 
established at the start. 



Phase Three — Process Assessment 

VJhile some steps in process assessment must be started quite early dur- 
ing the project, most cannot be started until sound input and product data 
are available, to work with. The process itself defines the functional rela- 
tionship between input (resources) and output (student performance). Process 
assessment is concerned with determining ways of modifying this fvinctional 
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relationship to reduce discrepancies identified through product assessment, 
within resource allocation constraints identified through input assessment. 
Steps In process assessment include the following: 

1. Identify those broad process areas which appear to relate to pro- 
duct discrepancies of interest (generally, the larger discrepancies). 

2. Using available research, delimit these broad areas to processes 
most reasonably indicative of producing positive change in outputs. 

3. Develop "process objectives" whicli are relevant to the product 
objectives to which the larger discrepancies relate. This task is reason- 
ably assigned to curriculum consultants and learning specialists. The "process 
objectives" will relate particularly to specific staff behavior, as opposed 

to the student behaviors. Contracts may be let witfi professionals to develop 
these objectives. 

4. Develop instruments to measure the extent to which process objectives 

are achieved. Item/ teacher sampling procedures may be utilized, where appropriate. 

5. Specify and develop an "interface" with product and input assessment. 
That is, when new processes are installed, the specific effects of these processes 
on student outputs and upon input allocation must be available for management 
decisions related to efficacy, efficiency, and cost-effectiveness. 

The development of process assessment procedures is necessary to obtain 
daita for interactioh with the other assessment phases. Only when these three 
aspects of the education enterprise can be measured and interrelated can sound 
professional judgments be reached. Kach depends upon the other, and a lack 
of any one phase obviates sound decisions relating to the other two. 



Establishing Accountability 

The systems approach lends itself well to the establishment of accountability. 
By stating precise outputs, specific tasks can be assigned to personnel, who can 
in turn be held accountable for the proper completion of those tasks. Within 
an operational framework, accountability for accomplishing the mission of each 
unit can thus be assigned to appropriate individuals. 

It is expected that the plan proposed here will lead to the establishment 
of accountability for educational outcomes. Actual achievement of outcomes 
must be determined by a comparison of product measures with pre-established 
standards (performance criteria) ♦ This task is most reasonably accomplished 
through "educational audit" procedures. However, the question arises as to the 
appropriateness of an educational audit conducted by those persons or agencies 
which established the performance standards initially. 

Several organizations exist which have the expertise and experience to 
conduct independent educational audits. While the plan itself does not call for 
a periodic audit procedure, audit is an indispensable management tool which 
will be utilized in providing the comparative information required by the plan. 
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Summary 

Improvement in education is not guaranteed by more money; neither is it 
guaranteed by the existence of product measures alone- Improvement implies 
positive change. Thus a knowledge of inputs, processes, and outputs together 
with statements of functional relationship:? between the three must be available 
to know (1) what change is needed; (2) how to bring about the change; and 
(3) the resources required for that change. 

The assessment plan proposed above focused on three phases of manageRTent 
control: input, process, and output. An attempt was made to show the inter- 
relationships inherent in this three-phase concept- Procedures for obtaining 
pertinent and relevant data during each phase were outlined- 

The brief report presented above is not a complete nor a specific pro- 
cedure for state educational assessment- The concept of this plan embodies a 
system approach utilizing three gross subsystems, each of which must be considered 
in a planned program for the improvement of education. 
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